A study assessing the characteristics of big data environments that predict high research impact: application of qualitative and quantitative methods

BACKGROUND: Big data offers new opportunities to enhance healthcare practice. While researchers have shown increasing interest to use them, little is known about what drives research impact. We explored predictors of research impact, across three major sources of healthcare big data derived from the government and the private sector. METHODS: This study was based on a mixed methods approach. Using quantitative analysis, we first clustered peer-reviewed original research that used data from government sources derived through the Veterans Health Administration (VHA), and private sources of data from IBM MarketScan and Optum, using social network analysis. We analyzed a battery of research impact measures as a function of the data sources. Other main predictors were topic clusters and authors’ social influence. Additionally, we conducted key informant interviews (KII) with a purposive sample of high impact researchers who have knowledge of the data. We then compiled findings of KIIs into two case studies to provide a rich understanding of drivers of research impact. RESULTS: Analysis of 1,907 peer-reviewed publications using VHA, IBM MarketScan and Optum found that the overall research enterprise was highly dynamic and growing over time. With less than 4 years of observation, research productivity, use of machine learning (ML), natural language processing (NLP), and the Journal Impact Factor showed substantial growth. Studies that used ML and NLP, however, showed limited visibility. After adjustments, VHA studies had generally higher impact (10% and 27% higher annualized Google citation rates) compared to MarketScan and Optum (p<0.001 for both). Analysis of co-authorship networks showed that no single social actor, either a community of scientists or institutions, was dominating. Other key opportunities to achieve high impact based on KIIs include methodological innovations, under-studied populations and predictive modeling based on rich clinical data. CONCLUSIONS: Big data for purposes of research analytics has grown within the three data sources studied between 2013 and 2016. Despite important challenges, the research community is reacting favorably to the opportunities offered both by big data and advanced analytic methods. Big data may be a logical and cost-efficient choice to emulate research initiatives where RCTs are not possible

Systematic differences between Cochrane and non-Cochrane meta-analyses on the same topic: a matched pair analysis

BACKGROUND: Meta-analyses conducted via the Cochrane Collaboration adhere to strict methodological and reporting standards aiming to minimize bias, maximize transparency/reproducibility, and improve the accuracy of summarized data. Whether this results in differences in the results reported by meta-analyses on the same topic conducted outside the Cochrane Collaboration is an open question. METHODS: We conducted a matched-pair analysis with individual meta-analyses as the unit of analysis, comparing Cochrane and non-Cochrane reviews. Using meta-analyses from the cardiovascular literature, we identified pairs that matched on intervention and outcome. The pairs were contrasted in terms of how frequently results disagreed between the Cochrane and non-Cochrane reviews, whether effect sizes and statistical precision differed systematically, and how these differences related to the frequency of secondary citations of those reviews. RESULTS: Our search yielded 40 matched pairs of reviews. The two sets were similar in terms of which was first to publication, how many studies were included, and average sample sizes. The paired reviews included a total of 344 individual clinical trials: 111 (32.3%) studies were included only in a Cochrane review, 104 (30.2%) only in a non-Cochrane review, and 129 (37.5%) in both. Stated another way, 62.5% of studies were only included in one or the other meta-analytic literature. Overall, 37.5% of pairs had discrepant results. The most common involved shifts in the width of 95% confidence intervals that would yield a different statistical interpretation of the significance of results (7 pairs). Additionally, 20% differed in the direction of the summary effect size (5 pairs) or reported greater than a 2-fold difference in its magnitude (3 pairs). Non-Cochrane reviews reported significantly higher effect sizes (P < 0.001) and lower precision (P < 0.001) than matched Cochrane reviews. Reviews reporting an effect size at least 2-fold greater than their matched pair were cited more frequently. CONCLUSIONS: Though results between topic-matched Cochrane and non-Cochrane reviews were quite similar, discrepant results were frequent, and the overlap of included studies was surprisingly low. Non-Cochrane reviews report larger effect sizes with lower precision than Cochrane reviews, indicating systematic differences, likely reflective of methodology, between the two types of reviews that could generate different interpretations of the interventions under question

Systematic differences between Cochrane and non-Cochrane meta-analyses on the same topic: a matched pair analysis

BACKGROUND: Meta-analyses conducted via the Cochrane Collaboration adhere to strict methodological and reporting standards aiming to minimize bias, maximize transparency/reproducibility, and improve the accuracy of summarized data. Whether this results in differences in the results reported by meta-analyses on the same topic conducted outside the Cochrane Collaboration is an open question. METHODS: We conducted a matched-pair analysis with individual meta-analyses as the unit of analysis, comparing Cochrane and non-Cochrane reviews. Using meta-analyses from the cardiovascular literature, we identified pairs that matched on intervention and outcome. The pairs were contrasted in terms of how frequently results disagreed between the Cochrane and non-Cochrane reviews, whether effect sizes and statistical precision differed systematically, and how these differences related to the frequency of secondary citations of those reviews. RESULTS: Our search yielded 40 matched pairs of reviews. The two sets were similar in terms of which was first to publication, how many studies were included, and average sample sizes. The paired reviews included a total of 344 individual clinical trials: 111 (32.3%) studies were included only in a Cochrane review, 104 (30.2%) only in a non-Cochrane review, and 129 (37.5%) in both. Stated another way, 62.5% of studies were only included in one or the other meta-analytic literature. Overall, 37.5% of pairs had discrepant results. The most common involved shifts in the width of 95% confidence intervals that would yield a different statistical interpretation of the significance of results (7 pairs). Additionally, 20% differed in the direction of the summary effect size (5 pairs) or reported greater than a 2-fold difference in its magnitude (3 pairs). Non-Cochrane reviews reported significantly higher effect sizes (P < 0.001) and lower precision (P < 0.001) than matched Cochrane reviews. Reviews reporting an effect size at least 2-fold greater than their matched pair were cited more frequently. CONCLUSIONS: Though results between topic-matched Cochrane and non-Cochrane reviews were quite similar, discrepant results were frequent, and the overlap of included studies was surprisingly low. Non-Cochrane reviews report larger effect sizes with lower precision than Cochrane reviews, indicating systematic differences, likely reflective of methodology, between the two types of reviews that could generate different interpretations of the interventions under question

Retrospective analysis of long-term gastrointestinal symptoms after Clostridium difficile infection in a nonelderly cohort.

Elderly patients and those with comorbid conditions are at high risk for poor outcomes after Clostridium difficile infection (CDI) but outcomes in a healthier, nonelderly population are not well described. We sought to investigate gastrointestinal diagnoses and CDI during hospitalizations in the 24 to 36 months after an initial episode of CDI in nonelderly patients in a cohort with an overall low prevalence of comorbid conditions. We performed a retrospective analysis of hospital admissions from 2010-2013 using the Truven MarketScan database of employment-based private insurance claims. Subjects = 18 years old); a CDI diagnosis in 2011 (index date); at least 12 months of pre-index continuous enrollment; and 24-36 months of continuous post-index enrollment were included. The 12 months of each subject's enrollment prior to the index date for a CDI served as the reference period for the analyses of that subject's post-CDI time periods. Hospital claims during the follow-up period were evaluated for gastrointestinal diagnoses and/or CDI ICD-9 codes. The risk of gastrointestinal diagnoses was assessed using Cox proportional hazards models adjusted for a pre-specified set of baseline demographic and clinical factors. During 2011, 5,632 subjects with CDI met the inclusion criteria for our study. The risk of gastrointestinal diagnoses in patients with a CDI diagnostic code for the same admission was almost 8-fold higher 3 months post-CDI (hazard ratio (HR) = 7.56; 95% confidence interval (CI): 2.97-19.19) than for subjects without CDI and remained statistically significant until month 24 (HR = 1.47; 95% CI = 1.04-2.08). After CDI, patients remained at risk for gastrointestinal symptoms with CDI for up to two years. There is an important, long-term healthcare burden after CDI in this population

Comparative effectiveness of different treatment pathways for opioid use disorder.

Question: What is the real-world effectiveness of different treatment pathways for opioid use disorder? Findings: In this comparative effectiveness research study of 40 885 adults with opioid use disorder that compared 6 different treatment pathways, only treatment with buprenorphine or methadone was associated with reduced risk of overdose and serious opioid-related acute care use compared with no treatment during 3 and 12 months of follow-up. Meaning: Methadone and buprenorphine were associated with reduced overdose and opioid-related morbidity compared with opioid antagonist therapy, inpatient treatment, or intensive outpatient behavioral interventions and may be used as first-line treatments for opioid use disorder

Overdose risk for veterans receiving opioids from multiple sources

OBJECTIVES: The aim of this study was to evaluate whether veterans in Massachusetts receiving opioids and/or benzodiazepines from both Veterans Health Administration (VHA) and non-VHA pharmacies are at higher risk of adverse events compared with those receiving opioids at VHA pharmacies only. STUDY DESIGN: A cohort study of veterans who filled a prescription for any Schedule II through V substance at a Massachusetts VHA pharmacy. Prescriptions were recorded in the Massachusetts Department of Public Health Chapter 55 data set. METHODS: The study sample included 16,866 veterans residing in Massachusetts, of whom 9238 (54.8%) received controlled substances from VHA pharmacies only and 7628 (45.2%) had filled prescriptions at both VHA and non-VHA pharmacies ( dual care users ) between October 1, 2013, and December 31, 2015. Our primary outcomes were nonfatal opioid overdose, fatal opioid overdose, and all-cause mortality. RESULTS: Compared with VHA-only users, more dual care users resided in rural areas (12.6% vs 10.6%), received high-dose opioid therapy (26.3% vs 7.3%), had concurrent prescriptions of opioids and benzodiazepines (34.8% vs 8.2%), and had opioid use disorder (6.8% vs 1.6%) (P \u3c .0001 for all). In adjusted models, dual care users had higher odds of nonfatal opioid overdose (odds ratio [OR], 1.29; 95% CI, 0.98-1.71) and all-cause mortality (OR, 1.66; 95% CI, 1.43-1.93) compared with VHA-only users. Dual care use was not associated with fatal opioid overdoses. CONCLUSIONS: Among veterans in Massachusetts, receipt of opioids from multiple sources was associated with worse outcomes, specifically nonfatal opioid overdose and mortality. Better information sharing between VHA and non-VHA pharmacies and prescribers has the potential to improve patient safety

Systematic differences between the Cochrane and non-Cochrane matched meta-analyses, in terms of a) natural log of effect size, and b) standard error of effect size.

<p>This figure regresses on a natural log scale pairs of Cochrane and non-Cochrane reviews in terms of effect size (Fig 3a) and standard error (Fig 3b). Each point on the scatter plot represents the intersection point of a Cochrane review with its matched pair in the non-Cochrane literature. In both cases, using T and F tests, the relationships are strongly correlated. However, in both, the slope of the line reveal that, on average, non-Cochrane reviews report slightly larger effect sizes but with larger standard errors (i.e., lower precision) than their matched Cochrane review.</p